Recently, AutoFlow has shown promising results on learning a training set for optical flow, but requires ground truth labels in the target domain to compute its search metric. Observing a strong correlation between the ground truth search metric and self-supervised losses, we introduce self-supervised AutoFlow to handle real-world videos without ground truth labels. Using self-supervised loss as the search metric, our self-supervised AutoFlow performs on par with AutoFlow on Sintel and KITTI where ground truth is available, and performs better on the real-world DAVIS dataset. We further explore using self-supervised AutoFlow in the (semi-)supervised setting and obtain competitive results against the state of the art.

对话中的情感认可（ERC）旨在检测给定对话中每种话语的情感。新提出的ERC模型利用了预培训的语言模型（PLM），并具有预训练和微调的范式，以获得良好的性能。但是，这些模型很少利用PLM的优势，并且对于缺乏明确的情感表达的对话而表现不佳。为了充分利用与话语中情感表达相关的潜在知识，我们提出了一种新颖的ERC模型Cisper，并使用新的及时和语言模型（LM）调整范式提出。具体而言，Cisper配备了及时融合与对话者的话语相关的上下文信息和常识，以更有效地实现ERC。我们的广泛实验表明，Cisper在最新的ERC模型中的出色表现以及利用这两种重要及时及时提高信息的有效性。为了方便地重现我们的实验结果，Cisper的Sourcecode和数据集已在https://github.com/deqingyang/cisper上共享。

Hierarchical text classification aims to leverage label hierarchy in multi-label text classification. Existing methods encode label hierarchy in a global view, where label hierarchy is treated as the static hierarchical structure containing all labels. Since global hierarchy is static and irrelevant to text samples, it makes these methods hard to exploit hierarchical information. Contrary to global hierarchy, local hierarchy as a structured labels hierarchy corresponding to each text sample. It is dynamic and relevant to text samples, which is ignored in previous methods. To exploit global and local hierarchies,we propose Hierarchy-guided BERT with Global and Local hierarchies (HBGL), which utilizes the large-scale parameters and prior language knowledge of BERT to model both global and local hierarchies.Moreover,HBGL avoids the intentional fusion of semantic and hierarchical modules by directly modeling semantic and hierarchical information with BERT.Compared with the state-of-the-art method HGCLR,our method achieves significant improvement on three benchmark datasets.

变形金刚正在改变计算机视觉的景观，特别是对于识别任务。检测变压器是对象检测的第一个完全结束的学习系统，而视觉变压器是用于图像分类的第一个完全变压器的架构。在本文中，我们集成了视觉和检测变压器（Vidt）以构建有效和高效的物体探测器。 VIDT引入了重新配置的注意模块，将最近的Swin变压器扩展为独立对象检测器，然后是计算高效的变压器解码器，该解码器利用多尺度特征和辅助技术来提高检测性能，而无需多大增加计算负载。 Microsoft Coco基准数据集上的广泛评估结果表明，VIDT在现有的基于变压器的对象检测器中获得了最佳的AP和延迟折衷，并且由于大型型号的高可扩展性而实现了49.2AP。我们将在https://github.com/naver-ai/vidt发布代码和培训的型号

We present a network architecture for processing point clouds that directly operates on a collection of points represented as a sparse set of samples in a high-dimensional lattice. Naïvely applying convolutions on this lattice scales poorly, both in terms of memory and computational cost, as the size of the lattice increases. Instead, our network uses sparse bilateral convolutional layers as building blocks. These layers maintain efficiency by using indexing structures to apply convolutions only on occupied parts of the lattice, and allow flexible specifications of the lattice structure enabling hierarchical and spatially-aware feature learning, as well as joint 2D-3D reasoning. Both point-based and image-based representations can be easily incorporated in a network with such layers and the resulting model can be trained in an end-to-end manner. We present results on 3D segmentation tasks where our approach outperforms existing state-of-the-art techniques.

Given two consecutive frames, video interpolation aims at generating intermediate frame(s) to form both spatially and temporally coherent video sequences. While most existing methods focus on single-frame interpolation, we propose an end-to-end convolutional neural network for variable-length multi-frame video interpolation, where the motion interpretation and occlusion reasoning are jointly modeled. We start by computing bi-directional optical flow between the input images using a U-Net architecture. These flows are then linearly combined at each time step to approximate the intermediate bi-directional optical flows. These approximate flows, however, only work well in locally smooth regions and produce artifacts around motion boundaries. To address this shortcoming, we employ another U-Net to refine the approximated flow and also predict soft visibility maps. Finally, the two input images are warped and linearly fused to form each intermediate frame. By applying the visibility maps to the warped images before fusion, we exclude the contribution of occluded pixels to the interpolated intermediate frame to avoid artifacts. Since none of our learned network parameters are time-dependent, our approach is able to produce as many intermediate frames as needed. To train our network, we use 1,132 240-fps video clips, containing 300K individual video frames. Experimental results on several datasets, predicting different numbers of interpolated frames, demonstrate that our approach performs consistently better than existing methods.

We present a compact but effective CNN model for optical flow, called PWC-Net. PWC-Net has been designed according to simple and well-established principles: pyramidal processing, warping, and the use of a cost volume. Cast in a learnable feature pyramid, PWC-Net uses the current optical flow estimate to warp the CNN features of the second image. It then uses the warped features and features of the first image to construct a cost volume, which is processed by a CNN to estimate the optical flow. PWC-Net is 17 times smaller in size and easier to train than the recent FlowNet2 model. Moreover, it outperforms all published optical flow methods on the MPI Sintel final pass and KITTI 2015 benchmarks, running at about 35 fps on Sintel resolution (1024×436) images. Our models are available on https://github.com/NVlabs/PWC-Net.

快速移动受试者的运动模糊是摄影中的一个长期问题，由于收集效率有限，尤其是在弱光条件下，在手机上非常常见。尽管近年来我们目睹了图像脱毛的巨大进展，但大多数方法都需要显着的计算能力，并且在处理高分辨率照片的情况下具有严重的局部动作。为此，我们根据手机的双摄像头融合技术开发了一种新颖的面部脱毛系统。该系统检测到主题运动以动态启用参考摄像头，例如，最近在高级手机上通常可用的Ultrawide Angle摄像机，并捕获带有更快快门设置的辅助照片。虽然主镜头是低噪音但模糊的，但参考镜头却很锋利，但嘈杂。我们学习ML模型，以对齐和融合这两张镜头，并在没有运动模糊的情况下输出清晰的照片。我们的算法在Google Pixel 6上有效运行，每次拍摄需要463毫秒的开销。我们的实验证明了系统对替代单片，多帧，面部特异性和视频脱张算法以及商业产品的优势和鲁棒性。据我们所知，我们的工作是第一个用于面部运动脱毛的移动解决方案，在各种运动和照明条件下，在数千个图像中可靠地工作。

培训细节和数据集对于筏等最新的光流模型有多重要？它们会概括吗？为了探索这些问题，而不是开发新的模型，我们将重新访问三个突出的模型，即PWC-NET，IRR-PWC和RAFT，并采用一组常见的现代培训技术和数据集，并观察到显着的性能增长，证明了重要性和普遍性这些培训细节。我们新训练的PWC-NET和IRR-PWC模型显示出惊人的改进，与Sintel和Kitti 2015 Benchmarks相比，最高30％的结果与原始发布的结果相比。他们的表现胜过2015年Kitti的最新流程1D，而推断过程中的速度快3倍。我们新训练的筏子在2015年的Kitti上获得了4.31％的成绩，比写作时所有已发表的光流方法更准确。我们的结果表明，分析光流方法的性能提高时，分离模型，训练技术和数据集的贡献的好处。我们的源代码将公开可用。

我们提出了一种框架插值算法，该算法从两个输入图像中综合了具有大型内部运动的两个输入图像。最近的方法使用多个网络来估计光流或深度以及专用于框架合成的单独网络。这通常是复杂的，需要稀缺的光流或深度地面真相。在这项工作中，我们提出了一个单一的统一网络，该网络以多尺度的特征提取器为特色，该特征提取器在各个尺度上共享权重，并且可以单独从框架中训练。为了综合酥脆和令人愉悦的框架，我们建议使用革兰氏矩阵损失来优化我们的网络，从而衡量特征地图之间的相关差异。我们的方法优于XIPH大型运动基准的最先进方法。与使用感知损失的方法相比，我们还可以在Vimeo-90K，Middlebury和UCF101上获得更高的分数。我们研究了体重共享和培训的效果，该数据集的运动范围不断增加。最后，我们证明了模型在综合高质量和临时连贯的视频中的有效性，该视频在具有挑战性的近乎修复的照片数据集中。代码和预训练模型可在https://film-net.github.io上找到。